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Abstract 


The ability of educational games to promote students’ 
engagement in learning and practice depends on perceived 
enjoyment of those games. This study investigated high 
school students’ perceptions and enjoyment of games within 
the Writing Pal intelligent tutoring system. In accord with 
research on motivation, results showed that perceived 
helpfulness and difficulty of the games were the main 
predictors of enjoyment, whereas graphics quality and 
writing apprehension were not. Perceived difficulty was 
most salient for generative practice games in which students 
had to apply strategies to write original text. 


Introduction 


Digital games have become a popular approach within 
educational technology due to their assumed motivational 
potential (Dondlinger, 2007; Young et al., 2012). Some 
learning technologies, including intelligent tutoring 
systems (ITSs), can require significant time investment for 
training and practice leading to student boredom or 
disengagement (McNamara, Jackson, & Graesser, 2009). 
Games are believed to leverage students’ enjoyment of 
gaming to promote engagement in such environments. 
However, individual games can differ remarkably in their 
design (e.g., narrative and rewards), and students likewise 
differ in their game preferences and attitudes toward the 
learning domain. If the success of educational games 
depends upon students’ game perceptions, then a crucial 
goal for educational technology research is to better 
understand how game design and student attitudes interact 
to influence enjoyment (Quick, Atkinson, & Lin, 2012). 

In this study, we examine how students’ enjoyment of 
educational games for writing varies based on writing 
apprehension and perceptions of game utility, difficulty, 
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and graphics. We conduct this research in the context of 
Writing Pal (W-Pal), an ITS that provides strategy 
instruction via a animated lessons, educational games, and 
writing practice with automated feedback (Roscoe & 
McNamara, in press; Roscoe, Varner, Weston, Crossley, & 
McNamara, in press). 


Enjoyment, Motivation, and Game Design 


For educational games to motivate students to play, and 
thereby engage in target learning tasks or acquire new 
strategies, the games must be perceived as enjoyable. The 
games must entice students to continue playing despite 
obviously embedded learning goals and expectations. 

Educational game researchers have proposed several 
frameworks characterizing either game players or game 
features with respect to enjoyment. Several taxonomies 
describe players’ goals for gaming, such as earning points, 
collecting treasures, exploring new worlds, discovering 
game mechanics, socializing, and defeating other players 
(Quick et al., 2012; Yee, 2006). Other taxonomies describe 
generalized game features that players may find enjoyable, 
such as fantasy, narrative, challenge, fellowship, discovery, 
expression, physical activity, altruism, and competition 
(Hunicke, LeBlanc, & Zubek, 2004; Winn, 2008). 
Altogether, educational game researchers have developed a 
lengthy list of factors that may promote enjoyment, 
although many of these taxonomies have not been 
empirically validated (Quick et al., 2012). 

Another limitation of such taxonomies is that they tend 
not to be grounded in formal motivational theories (Ryan, 
Rigby, & Przybylski, 2006), but instead are derived from 
observation or self-reported preferences. Nevertheless, 
there is some overlap between motivational theories and 
gaming taxonomies, such as perceptions of one’s 
capabilities and improvement (e.g., earning points and 
achievements), social interactions (e.g., competition, 
fellowship, and role-playing), and personal goals. 


For example, expectancy-value theories posit that 
individuals’ engagement in learning tasks depends upon 
task-value beliefs and expectations of success (Wigfield & 
Eccles, 2000). Students are more likely to engage in tasks 
that appear beneficial and offer opportunities for success 
and growth. Similarly, self-determination theory argues 
that enjoyment arises when three needs are met: 
competence, autonomy, and relatedness (Ryan et al., 
2006). Students are more likely to enjoy and persist in 
tasks that foster feelings of challenge, capability, and 
choice, and offer social interaction. Finally, achievement 
goal theories (Harackiewicz, Barron, Pintrich, Elliot, & 
Thrash, 2002) emphasize how students’ decisions are 
driven by their goals for achievement, such as competing 
favorably with others or mastering the subject for the sake 
of personal satisfaction. Together, these theories suggest 
that students’ attitudes and expectations in the learning 
domain, and their perceptions of the games as helpful for 
learning or developing competence, may be important 
predictors of students’ enjoyment of educational games. 

Quick et al. (2012) have argued that it is important to 
“consider game design and player perceptions in tandem, 
because both are integral parts of game experiences” (p. 
12). In this study, we examine how students’ anxiety 
toward writing, along with perceptions of game utility and 
difficulty, influence their enjoyment of educational games 
for writing strategies. We then consider how relations 
among these factors may differ based on the nature of the 
writing games and tasks. 


Game-based Strategy Practice in Writing Pal 


W-Pal is an ITS developed to support adolescent students’ 
acquisition of writing strategies across three phases of the 
writing process (Roscoe et al., in press). Eight instructional 
modules are included, which cover strategies for prewriting 
(Freewriting and Planning), drafting (ntroduction 
Building, Body Building, and Conclusion Building), and 
revising (Paraphrasing, Cohesion Building, and Revising). 
Students are introduced to specific strategies via short 
animated videos narrated by pedagogical agents. W-Pal 
also allows students to practice these strategies by writing 
prompt-based, persuasive essays. Natural language 
processing (NLP) algorithms generate holistic quality 
ratings and guide formative feedback on students’ strategy 
use (McNamara, Crossley, & Roscoe, 2012). 

A unique aspect of W-Pal is that it incorporates a suite 
of 16 games (see Table 1) that target specific strategies. W- 
Pal games offer an intermediate level of practice in which 
students can focus on a few strategies in isolation before 
trying to apply them simultaneously in a single essay. 
Students are typically asked to view the lessons, play the 
practice games, and then write practice essays for each of 
the modules. Cognitively, this breakdown of essay 
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composition into manageable sub-goals is important 
because novice writers often struggle to coordinate the 
multiple tasks of the writing process (Breetvelt, van den 
Bergh, & Rijlaarsdam, 1994). Prior analyses have found 
that students’ performance in W-Pal games positively 
predicted learning of writing strategies (Roscoe, Brandon, 
Snow, & McNamara, in press). Students who earned 
higher scores in the games were better able to articulate 
strategies for planning, drafting, and revising essays. 


Table |. Brief Descriptions of Writing Pal Practice Games 


Game (Module) Description 

Freewrite Flash Fill the Idea Meter and earn Idea Flash 
(Freewriting) Cards by freewriting on a prompt. 

Mastermind Outline Repair the Mastermind Mainframe by 
(Planning) assembling an outline from given 


argument and evidence statements 
Travel to various destinations and earn 
souvenirs by selecting appropriate 
arguments and evidence. 


Planning Passage 
(Planning) 


Dungeon Escape Escape by avoiding the guard and rising 


(Introductions, waters. Select doors by labeling 
Conclusions) attention-grabbing techniques. 

Essay Launcher Rescue spaceships by selecting thesis 
(Introductions) statements and attention-grabbers for 

sample introduction paragraphs. 

Fix It Evaluate paragraphs for missing key 
(Introductions, Body, elements, such as thesis statements and 
or Conclusions) evidence. Fix the broken circuit board. 

RoBoCo Build robots by writing topic and 
(Body) evidence sentences for a given thesis. 

Lockdown Stop computer hackers by writing 
(Conclusions) conclusions based on a given outline. 


Adventurer’s Loot 
(Paraphrasing) 


Explore different locations and obtain 
treasure by correctly identifying use of 
paraphrasing strategies. 


Map Conquest Earn flags by identifying paraphrasing 


(Paraphrasing) strategies, and then use those flags to 
conquer the game board. 
Undefined & Mined Disarm mines by identifying undefined 
(Cohesion) referents in short texts. 
CON-Artist Catch a thief by following clues. The 
(Cohesion) clues are solved by selecting transition 


words to link given sentences. 


Speech Writer 
(Revising) 


Help a friend on the debate team revise 
a speech. Identify the major problems 
and then edit the speech to improve it. 


W-Pal games exhibit a variety of designs. For example, 
one salient dimension is the presence of narrative. In Essay 
Launcher, students must rescue spaceships and guide them 
to Earth. To do so, students select an appropriate thesis 


statement for an introduction paragraph (“repair the ship”) 
and identify the attention-grabbing technique used (“set the 
course”). In Lockdown, students take on the role of an 
agent for the Writing Intelligence Agency, and must help 
“protect essays” from hackers by writing conclusion 
paragraphs based on an outline. NLP algorithms assess 
paragraph quality and determine whether the student- 
generated conclusion is in “danger,” “at risk,” or “secure.” 

Other games lack a strong narrative focus. Undefined & 
Mined helps students practice cohesion building by 
identifying undefined referents. Students read texts and 
click on undefined terms, each of which is associated with 
a mine. Correct selections disarm a mine, but mistakes 
cause an explosion. In Fix /t, students evaluate paragraphs 
for flaws, such as missing evidence. Students select the 
problem in the Regular Round and then select text to fix 
the problem in a Bonus Round. Correct fixes earn a Golden 
Circuit. In the final phase, students repair a circuit board 
via a Sudoku-like puzzle, and Golden Circuits are used to 
automatically fill in one correct answer. There are three 
versions of Fix It: Introduction Building (IB), Body 
Building (BB), and Conclusion Building (CB). 

Another key dimension for W-Pal games is the nature of 
the practice task. Many games embed an identification task 
in which students label strategy exemplars or problems that 
could be improved using a strategy. For example, Essay 
Launcher involves the identification of thesis statements 
and attention-grabbers, and Undefined & Mined requires 
identification of undefined words. In Planning Passage, 
students take a “roadtrip” and unlock destinations and 
souvenirs by identifying applicable argument and evidence 
sentences. In Dungeon Escape, students escape a dungeon 
while avoiding rising waters and a guard. To unlock doors 
and get to the exit, students identify the attention-grabbing 
techniques (Introduction Building) or attention-holding 
techniques (Conclusion Building) exhibited in a paragraph. 

Other games in W-Pal involve generative practice in 
which students author original text or assemble an outline. 
Lockdown is one such generative game. In Speech Writer, 
students help a friend prepare for an upcoming debate by 
revising an essay. Students first identify problems in the 
essay, and then try to edit the essay to fix the problems. 
NLP algorithms assess whether the revised essay improves 
along various dimensions from the original essay, which 
determines the final score. Speech Writer combines both 
identification and generative practice, with greater 
emphasis on the generative aspects. In RoBoCo, students 
are placed in the role of an engineer at the Robot Body 
Company who must develop new robot models. Students 
earn robot heads by writing topic sentences and robot 
bodies by writing evidence sentences. The student- 
generated texts are rated via NLP algorithms that 
determine how many robot parts are awarded. After two 
rounds of writing, assemble earn parts (if any) into robots. 
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The final robots are displayed on stage during the Annual 
Show. The final score (expressed as profit for the 
company) is based on text quality and number of robots. 

An important facet of W-Pal design is that students 
interact with all of the W-Pal games during training. Thus, 
rather than rating individual, isolated games, students can 
play and evaluate multiple games within the same context. 
Thus, comparisons can be made across games and game 
features. For this study, our questions were 1) How do 
perceptions of game utility, difficulty, and graphics, along 
with students’ prior writing attitudes, influence their 
enjoyment of educational games? 2) Do relationships 
among enjoyment, game perceptions, and writing attitudes 
differ across individual games or game types? 


Method 


Participants 


High school students (n = 65) from a large, urban area in 
the Southwest United States participated in a lab-based 
summer program using W-Pal. Average age was 16, with 
70.8% female students. With regards to ethnicity, 6.2% of 
students identified as African-American, 15.4% as Asian, 
24.6% as Caucasian, and 44.6% as Hispanic. Average 
grade level was 10.2 with 35.4% of students reporting a 
GPA of 3.0 or below. 


Procedure 


Students attended 10 sessions over 2-4 weeks. The first and 
final sessions included tests of reading comprehension, 
vocabulary, writing proficiency, strategy knowledge, and 
writing attitudes. Sessions 2-9 were devoted to training. 
Some students (n = 33) used the full W-Pal, including 
essay writing, lessons, and games. In each session, these 
students wrote and revised one essay and completed one 
module. Students rated each game immediately after 
playing. A comparison group interacted only with the 
essay and feedback tools (n = 32). These students wrote 
and revised two essays per session with feedback. For the 
current paper, only game perceptions and writing attitudes 
from the W-Pal condition students are discussed here. 


Measures 


Game-perception Surveys 

Students used a 4-point scale to rate games on enjoyment, 
helpfulness for learning, ease of gameplay, and graphics. 
Higher ratings indicated a more positive response. Two 
additional questions were asked as a check to make sure 
the games were playable. Overall, students considered 
game instructions to be understandable (M = 3.4, SD = 0.4) 
and game controls to be clear (M = 3.5, SD = 0.4). 


Writing Apprehension Test 

Students’ attitudes toward writing and their own writing 
abilities influence their enjoyment of writing and their 
writing performance (e.g., Daly & Miller, 1975; Pajares, 
2003). The Daly-Miller Writing Apprehension Test (WAT) 
is a 26-item measure in which students rate their agreement 
to statements about writing on a 5-point scale (Daly & 
Miller, 1975). Students respond to both positively-worded 
(“Writing is a lot of fun”) and negatively-worded items 
(“I’m not good at writing”). Higher WAT scores indicate 
greater comfort with writing whereas lower scores indicate 
more apprehension. Prior research with the measure has 
shown high reliability (7 = .92-.94). 


Results 


Overall Attitudes, Perceptions, and Enjoyment 


Average WAT scores (M = 63.0, SD = 18.3) suggested that 
many students experienced a moderate to high level of 
writing apprehension. Indeed, many students (40%) scored 
below 60, indicating strong anxiety. Despite apprehension 
about writing, students rated the W-Pal games as enjoyable 
(M = 3.0, SD = 0.5). Games were also perceived as helpful 
(M = 3.1, SD = 0.5), somewhat easy to play (M = 2.8, SD = 
0.5), and graphically appealing (VM = 3.3, SD = 0.5). 

A regression analysis was conducted to examine how 
students’ attitudes and perceptions influenced game 
enjoyment. Enjoyment was not significantly correlated 
with WAT scores (r = .20, p = .26), but was significantly 
correlated with perceived helpfulness (r = .88, p < .001), 
ease (r = .52, p < .001), and graphics (r = .65, p < .001). 
All four variables were entered as predictors of enjoyment 
in a linear regression. The overall model was significant, 
F(4,28) = 42.48, p < .001, with an R’ of .86. Only 
helpfulness (f = .76, t = 7.00, p < .001) and ease (£ = .30, t 
= 7.00, p < .001) were significant predictors of enjoyment. 
Writing apprehension (f = -.04) and perceptions of game 
graphics (£ = .07) were not significant predictors. 

Overall, students’ writing anxiety did not seem to hinder 
enjoyment of the games, which provides further support 
for the motivational potential of games. Similarly, 
enjoyment was not influenced by game graphics. This 
result is encouraging because it suggests that educational 
games need not include 3D environments with lifelike 
graphics. However, graphics ratings were very similar 
across games. Thus, this result should be further explored 
with games that are more variable in graphic quality. 

Enjoyment of W-Pal games seemed to be most related to 
perceived helpfulness. As one might expect based on 
motivational theories (e.g., Ryan et al., 2006; Wigfield & 
Eccles, 2000), games that are perceived to be beneficial 
may better engage students’ interest. Similarly, perceived 
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ease also contributed to enjoyment. Students are likely 
drawn to games in which they can experience competence, 
perhaps building their self-efficacy for writing (Ryan et al., 
2006). Together, these findings suggest that w¢ility may be 
a key factor for educational game design. Rather than on 
focusing on graphics or other “bells and whistles,” it may 
be crucial for students to appreciate how the game will 
help them improve. It may be difficult to fully disguise 
many learning tasks as games, but perhaps such disguises 
are not necessary if students value what the games have to 
offer educationally (while also being reasonably fun). 


Individual Game Ratings and Perceptions 


The preceding analyses collapsed ratings across all W-Pal 
games. However, individual games differ with respect to 
particular features, mechanics, and demands. To explore 
how these differences influenced predictors of enjoyment, 
mean ratings were obtained for each game (Table 2) and 
separate regressions (Table 3) were conducted. 


Table 2. Mean Enjoyment and Game Perception Ratings. 


Game Enjoy H E G 
Freewrite Flash 3.1 3.2 2.6 3.2 
Mastermind 3.3 34 2.8 3.2 
Planning Passage 2.8 32 29 3.2 
Essay Launcher 2.4 2.9 26 3.3 
Dungeon Esc. (IB) 323 3.3 32 3.4 
Fix It (IB) 3.2 3.2 3.1 3.3 
RoBoCo 22 25 2.0 3.3 
Fix It (BB) 3.2 33 33 34 
Lockdown 2.7 28 24 3.3 
Dungeon Esc. (CB) 2.9 2.9 2.7 33 
Fix It (CB) 3.2 34 33 3.4 
Adventurer’s Loot 3.1 3.1 3.0 3.4 
Map Conquest 3.2 3.3 2.8 3.4 
Und. & Mined 3:2 322.9 3.3 
CON-Artist 3:5 35 33 34 
Speech Writer 2.5 2.8 24 3.3 


Note: H = helpfulness, E = ease, and G = graphics. 


Overall, students appeared to enjoy most of the games 
(i.e., most means > 2.5). However, a few games stood out 
as somewhat less enjoyable than others: RoBoCo, Speech 
Writer, Lockdown, and Essay Launcher. These games also 
tended to receive somewhat lower ratings with regard to 
helpfulness and ease. These results parallel previous 
observations showing that helpfulness and ease of 
gameplay predicted enjoyment across games. In this case, 
individual games that were perceived as more challenging 
and less helpful were rated as somewhat less enjoyable. 

Regressions were conducted to test how helpfulness, 
ease, graphics, and writing apprehension differentially 


predicted enjoyment for individual games (Table 3). All of 
the regressions were significant (i.e., p-values < .01 or 
.001). Helpfulness ratings were a significant, positive 
predictor of enjoyment for 15 of the games, and were the 
only predictor of enjoyment for half of the games. Ease 
was a positive predictor for six games and graphics appeal 
was a positive predictor for four games. Writing 
apprehension did not predict enjoyment for any game. 


Table 3. Significant Predictors of Game Enjoyment. 


Game R F Predictors 
Freewrite Flash 57 9.15 H 
Mastermind Outline 57 8.92 H E 
Planning Passage 58 9.68 H 

Essay Launcher 94 106.57 H G 
Dungeon Esc. (IB) 55 8.64 H 

Fix It (IB) 51 6.97 H 

RoBoCo .73 18.25 H E 

Fix It (BB) .74 18.97 H 
Lockdown -76 22.53 H E G 
Dungeon Esc. (CB) 44 5.39 H 

Fix It (CB) 58 9.59 G 
Adventurer’s Loot 38 4.33 H E 

Map Conquest =) 8.21 H 

Und. & Mined -76 21.77 H E 
CON-Artist 58 9.47 H 

Speech Writer 79 26.06 H E G 


Note: H = helpfulness, E = ease, and G = graphics. Games in 
which ease of play predicted enjoyment are highlighted in bold. 


Discriminating Game Features 


To better understand how features influenced relations 
among enjoyment, helpfulness, and ease of gameplay, we 
contrasted the six games for which both helpfulness and 
ease were predictors of enjoyment (Mastermind Outline, 
RoBoCo, Lockdown, Adventurer’s Loot, Undefined & 
Mined, and Speech Writer) with the nine games for which 
helpfulness was a predictor but ease was not a predictor. 
Overall, the nine “Help-Only Games” were rated as 
more enjoyable, more helpful, and easier than the six 
“Help/Ease Games.” One straightforward interpretation of 
these results is that greater game challenge increased the 
salience of difficulty in students’ perceptions of games. 
When games are seen as easy, then students’ enjoyment of 
the game may be primarily a function of whether the task 
is beneficial or useful for learning. However, as students 
experience more challenge or failure, perceived difficulty 
begins to influence enjoyment, perhaps due to frustration. 
This interpretation is supported by motivation research, 
which argues that utility and competence affect task 
enjoyment (Ryan et al., 2006; Wigfield & Eccles, 2000). 
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Table 4. Mean Enjoyment, Helpfulness, and Ease for 
Games Based on Regression Pattern Grouping. 


Game Grouping 


Rating Help-Only Help/Ease (32) Pp 

Enjoyment 3.1 (51) 2.8 (.54) 3.41 002 
Helpfulness 3.1 (.50) 2.9 (.57) 3.41 .002 
Ease of play 2.9 (.55) 2.6 (.59) 4.82 <.001 


With respect to game features, the primary difference 
appeared to be the nature of the learning task. Specifically, 
four of the Help/Ease Games involved generative practice 
(Mastermind Outline, RoBoCo, Lockdown, and Speech 
Writer) in which students had to author original text or an 
outline. In Adventurer’s Loot, students must analyze an 
original sentence, and then determine which of four target 
sentences instantiates a specified paraphrasing strategy. In 
Undefined & Mined, students must carefully study a text to 
determine which pronouns and referents are defined or 
undefined based on the textual context. These games 
require thoughtful application of writing strategies; 
perceptions of these games as more difficult were accurate. 

Interestingly, traditional game features mentioned in 
game taxonomies did not seem to discriminate among the 
Help-Only and Help/Ease Games. Both groups included 
narrative (e.g., RoBoCo and Dungeon Escape) and non- 
narrative games (e.g., Undefined & Mined and Fix It), and 
games with fantasy elements (e.g., RoBoCo, Adventurer’s 
Loot, Essay Launcher, and Dungeon Escape). Both groups 
also included games with simple mechanics in which each 
round featured similar game play (e.g., Essay Launcher, 
and Undefined & Mined) and cases in which game play 
varied across different phases (e.g., Fix It and Speech 
Writer). In sum, for educational games incorporated into 
W-Pal, traditional game features seemed less important to 
enjoyment than did the perceived utility of the games. 


Discussion 


Given the motivational potential of educational games, it is 
important to explore how and why students enjoy games 
for learning. In this study, high school students interacted 
with practice games in the W-Pal tutor. Students perceived 
these games to be enjoyable, useful for learning, easy to 
play, and graphically appealing. The central finding was 
that the helpfulness of the games for learning strategies 
was the strongest and most consistent predictor of 
enjoyment. Students enjoyed the games more when they 
felt the games helped them improve their writing. 
However, as students experienced greater difficulty with 
generative practice, perceived enjoyment and helpfulness 
somewhat decreased. These findings parallel research on 
motivation and learning, which emphasizes the role of 


utility and competence expectations (Ryan et al., 2006). 
Students’ enjoyment of educational games may be a 
balancing act between whether the game seems to help or 
hinder learning progress (Easterday, 2011). Factors that 
relate to enjoyment of entertainment games (e.g., narrative 
and exploration) may be less salient in educational settings. 

One limitation was the sample size of students using W- 
Pal games (n = 33). This constraint was necessary because 
each student required 10 sessions (about 3 weeks) to finish 
the study. Another limitation is that students could not play 
each game multiple times during that period. It is plausible 
that students’ game perceptions would evolve over time. 
Specifically, easier games may become boring as students 
master them; harder games may become more enjoyable as 
students overcome initial failures and notice improvement 
in their scores and strategy use. Temporal aspects of game 
perceptions could not be assessed in the current study. 

For ITS developers who wish to incorporate educational 
games to promote engagement, these findings suggest two 
recommendations. First, the utility of the games should be 
made salient to students, perhaps through mechanics or 
feedback that highlight students’ progress and improved 
performance. Positive feedback can improve the efficacy 
of ITSs (Mitrovic, Ohlsson, & Barrow, 2013), and such 
feedback could guide students’ attention to game benefits 
and promote feelings of competence leading to enjoyment. 
Second, differences in game enjoyment may inform 
methods for adaptive game selection and design. Based on 
students’ levels of engagement and performance, student 
models may guide learners along different progressions. 
Some students may need to play identification games (or a 
“tutorial mode”) first, to build confidence and skill, and 
only then advance to generative practice. However, skilled 
students may play difficult games from the outset. Starting 
with easy games might be less engaging and perhaps 
hinder motivation for higher-performing students. 
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